DFG Priority Programme SPP 1736: Algorithms for Big Data

Proposal to establish a DFG Priority Program

Design of Algorithms for Big Data Analytics

Very large volumes of data are being collected in every domain of human endeavour. The term Big Data has become pervasive recently and denotes the problems related to the storage, management, and analysis of these large amounts of data. Business, sciences, and engineering are all becoming dependent on their ability to harness useful knowledge from very large datasets. For example, the research ...

Adaptive Caching Algorithms for Big Data Systems

Today’s Big Data platforms have enabled the democratization of data by allowing data sharing among various data processing frameworks and applications that run in the same platform. This data and resource sharing, combined with the fact that most applications tend to access a hot set of the data has led to the development of external, in-memory, distributed caching frameworks. In this paper, we...

MapReduce Algorithms for Big Data Analysis

There is a growing trend of applications that should handle big data. However, analyzing big data is a very challenging problem today. For such applications, the MapReduce framework has recently attracted a lot of attention. Google’s MapReduce or its open-source equivalent Hadoop is a powerful tool for building such applications. In this tutorial, we will introduce the MapReduce framework based...

Parallel Algorithms for Big Data Optimization

We propose a decomposition framework for the parallel optimization of the sum of a differentiable function and a (block) separable nonsmooth, convex one. The latter term is usually employed to enforce structure in the solution, typically sparsity. Our framework is very flexible and includes both fully parallel Jacobi schemes and Gauss-Seidel (i.e., sequential) ones, as well as virtually all pos...